Still to ARRIVE at adequate reporting of orthodontic studies involving animal models

Abstract Background The ARRIVE 2.0 guidelines were introduced to improve the reporting of animal studies. The aim of this study was to assess the reporting adherence of orthodontic speciality animal studies in relation to ARRIVE 2.0 guidelines. Associations between the reporting and study characteristics were explored. Materials and method An electronic database search was undertaken using Medline via PubMed (www.pubmed.ncbi.nlm.nih.gov) to identify studies meeting the eligibility criteria published between 1 January 2018 and 31 December 2023. Data extraction was performed in duplicate and independently. Descriptive statistics and frequency distributions for the responses to each checklist item were calculated. Mean values for adequate reporting per ARRIVE item were calculated. A sum score was calculated by adding the responses (0 = not reported, 1 = inadequate reporting, 2 = adequate reporting) per item and sub-questions. On an exploratory basis, univariable linear regression between summary score and study characteristics (year of publication, continent of authorship, type of centre, and number of authors) was performed. Results Three hundred and eighty-four studies were analysed. Variability in the adequate reporting of the ARRIVE 2.0 guideline items was evident. In particular, in 32% of studies, there was a lack of reporting of the priori sample size calculation. Overall, the mean reporting score for the sample was 57.9 (SD 6.7 and range 34–74). There were no associations between score and study characteristics except for a weak association for year of publication with a small improvement over time (each additional year). Conclusions The reporting of animal studies relevant to the speciality of orthodontics is sub-optimal in relation to the ARRIVE 2.0 guidelines. There was a tendency for the non-reporting of items pertaining to study sample size, eligibility, methods to reduce bias and interpretation/scientific implications. Greater awareness and reporting adherence to the ARRIVE 2.0 guidelines are required to reduce research waste involving animal models.


Introduction
Despite the introduction of computer models and tissue and cell cultures within scientific research, studies involving animal models remain a viable and important research resource [1].This is highlighted by the fact that in 2005 over twelve million animals were used for experiments in Europe [2].Studies involving animal models are commonly utilized to assess the safety and efficacy of interventions, where the results are then extrapolated to understand the possible mechanism of action of the intervention in humans [3].The validity of this approach has been questioned [4], as extrapolation of results on a 1:1 basis with human samples is hindered by variation in study design, bias, confounders, and innate differences in the underlying physiology between human and animal species.Furthermore, animal research is utilized when the results must be obtained by sacrifice of the animal and further histological investigation.The benefits of animal studies in orthodontics have been highlighted [5].For instance, using the mice model, the use of local RANKL gene transfer has been reported to accelerate tooth movement and hence has the potential to reduce treatment duration [6].
It has been reported that animal studies suffer from poor reproducibility which can ultimately affect the validity and impact of the findings [7,8].Errors and incomplete reporting of key components of study methodology leading to heterogeneity between animal studies have been highlighted [9].Importantly omissions in reporting hinder the translatability of animal research to humans [10,11].This is highlighted by the fact that the reported effects of an intervention in animal studies tend to be greater than those reported in human studies [12].However, the lack of homogeneity in study design and data analysis precludes direct comparisons between both study types [12].It is commonly accepted that Randomised Clinical Trials (RCTs) are considered the optimal study design to investigate clinical efficacy, effectiveness, and safety of interventions.Although the aims and objectives of animal studies may differ from those of randomized clinical trials, reporting of both study types should be subject to the same level of scrutiny and rigor [13].
To facilitate clear and transparent reporting of animal studies the ARRIVE guidelines were initially developed and introduced in 2010 [14] and subsequently updated and superseded by the ARRIVE 2.0 guidelines in 2020 [15].The original guidelines consisted of 20 items whereas the updated version is split into ten essential and 11 recommended items.The authors of the updated guidelines state they 'reordered items and split them into two sets based on their importance to assess the reliability of the study.There is no ranking within each set, items are ordered logically' [15].The updated ARRIVE 2.0 guidelines cover aspects such as study design, methodology, animal welfare, interpretation, and generalisability of results [15].New items added comprise of stating the inclusion and exclusion criteria, protocol registration, analysis plan, and data access [15].Previous studies assessing reporting adherence to the original ARRIVE guidelines have identified several areas of deficient reporting including items pertaining to a description of the housing and husbandry, information regarding the sample size calculation, and allocation of animals to experimental groups [16,17].The introduction of the new ARRIVE 2.0 guidelines has led to better reporting, but further improvement is still required [18].
As far as we are aware, an assessment of the reporting of animal studies published in the orthodontic literature has not been previously undertaken.The aim of this investigation was to assess the reporting adherence of animal studies relevant to the speciality of orthodontics in relation to ARRIVE 2.0 guidelines.Associations between the reporting of studies and study characteristics were explored.

Eligibility criteria
Studies involving any type of animal model were eligible for inclusion in this assessment.Studies published in English were only sourced.Studies involving human participants, systematic reviews, randomized clinical trials, prospective studies, case reports, pilot studies, and protocols were excluded.

Search for relevant articles
An electronic database search was undertaken using Medline via PubMed (www.pubmed.ncbi.nlm.nih.gov).One author (J.S.) performed a literature search of this database using medical subject headings and free-text terms related to 'orthodontic'.The following search filters were applied in the search: articles published between 1 January 2018 and 31 December 2023, English language only and involving animal models.The title, abstract and methodology section of each study were screened for the reported use of animal models.The applicability of the study to the orthodontic speciality was agreed by discussion between two assessors (D.F. and J.S.).

Selection and data extraction
The titles, abstracts, and full text of articles meeting the eligibility criteria were assessed independently by two assessors (D.F. and J.S.).Disagreements were discussed between both assessors (D.F. and J.S.) and resolved by a third assessor if required (N.P.).Data from each eligible animal study were extracted by both assessors (D.F. and J.S.) independently.Any disagreements were discussed between both assessors and resolved by a third assessor (N.P.).A standardized and prepiloted data extraction form was used.Prior to data extraction, both assessors (D.F. and J.S.) individually undertook an initial pilot calibration.The results were discussed between both assessors (D.F. and J.S.).Any disagreements were resolved by discussion a third assessor (N.P.).100% agreement was achieved.
At the study level, the following study characteristics were extracted: year of publication, journal title, continent of corresponding author (Europe, Americas and Asia, and other), number of authors, centre (single or multi), statistical significance of primary outcome (yes or no), and type of animal model.The reporting adherence of each animal study was assessed in relation to the ARRIVE 2.0 guidelines [15] (Supplementary Table I).This checklist consists of two parts: essential 10 and the recommended set (11 items), totalling 21 items.Further sub-type questions are required in 13/21 checklist items.The reporting of each item was assigned to one of three categories (0 = not reported, 1 = inadequate reporting and 2 = adequate reporting).To ensure consistency in the interpretation of each checklist item by the assessors, the pre-defined description of each item was directly referred to during data extraction.Furthermore, each item and their subtype questions were scored based on applicability.Each article and supplementary data files were also screened to determine if the authors had explicitly stated if they had reported their study (yes or no) in relation to the ARRIVE guidelines.

Statistical analysis
Descriptive statistics and frequency distributions for the responses to each checklist item were calculated.Mean values for adequate reporting per ARRIVE item were calculated and plotted using a radial plot.A sum score was calculated by adding the 0/1/2 per item response per question.On an exploratory basis, univariable linear regression between summary score and the following study characteristics was performed: year of publication, continent of authorship, type of centre, and number of authors.All analyses were conducted using the R programming language version 4.3.1 (Vienna, Austria).

Results
The initial search identified 2310 studies, of which 384 were deemed eligible for inclusion in this study (Fig. 1) (Supplementary Table II).The most frequent years of publication were 2020 (21.3%) and 2022 (21.1%).Corresponding authors of animal model studies were likely to be based in Asia and other countries (69.2%).Studies were typically multicentre (80.0%) and reporting a significant finding for the primary outcome.The median number of authors was 7 (IQR 4).The rat (61.9%) followed by mice (24.7%) were commonly used as the animal models (Table 1).
The distribution of scores per category (0 = not reported, 1 = inadequate reporting, and 2 = adequate reporting) for the ARRIVE 2.0 items are presented in Table 2. Items that tended not to reported include those pertaining to study sample size (item 2 sub-item b [32%]), eligibility (item 3 subitem a [33.3%] and b [41.1%]), blinding (item 5) (76.8%), results (sub-item 10 b 57.0%), animal care and monitoring The mean score for the sample was 57.9 (SD 6.7 and range 34-74).The mean score for the essential and recommended set items was 34.2 (SD 64.6 and range 17-44) and 23.8 (SD 3.5, range 13-32), respectively.Between journal titles, the mean score varied but interpretation of these scores needs to consider the number of studies published in each journal.For instance, the journal ACS Biomaterials Science & Engineering, only published one study which scored a mean score of 63.In contrast, American Journal of Orthodontics and Dentofacial Orthopaedics published 29 studies and the mean score was 60.9 (Supplementary Table III).
Table 3 shows the estimates, 95% confidence intervals and P-values for the effect of study characteristics on the ARRIVE 2.0 score.There were no associations between score and study characteristics except a weak association for year of publication with a small improvement over time: 0.41 for each additional year which translates to an average score increase by 2 units from the first to the last year of publication of the included studies.

Discussion
The ARRIVE guidelines were introduced to improve the reporting of animal studies [15].It is important to recognize that this distinct from assessing the quality of a study [8].However, there are also wider and more ethical considerations as by improving reporting, the guidelines aim to improve the quality of life of animals, quality of animal experiments, and reduce the need for animals to be used in research [18].Within this sample of animal studies published in the wider literature, the adequate reporting of certain ARRIVE 2.0 items was lacking in particular domains.These findings support the notion that reporting of key aspects in animal studies acts as one of the most common barriers against study reproducibility [19,20].Items that tended not to be reported include those pertaining to study sample size, eligibility, methods to reduce bias such as blinding, interpretation/scientific implications, and data access.Parallels can be drawn from previous studies which have reported similar findings [16,17].However, it should be borne in mind that these studies assessed reporting adherence in relation to the original ARRIVE guidelines.Although blinding is an important methodological step to reduce bias, its applicability to orthodontic in-vitro animal model studies needs to be considered for each study.For instance, PCR analysis of oral tissue may not require the need for blinding of the observer.
In almost a third of studies, there was a lack of clear reporting of the priori sample size calculation.It is well established that studies with animals tend to have small samples [21,22].When the sample is small, questions are raised as to the ability of the study to detect differences between the intervention or control/alternative intervention groups.The issue is further compounded by the lack of appropriate power/ sample size calculations.In this scenario, the risk of detecting a Type I error (false positives) against a null hypothesis of no treatment effect is increased.Therefore, these studies will be prone to reporting overestimated treatment effects [12].This has been highlighted in an interesting study, where the authors compared the reported treatment effects in animal and human studies for the non-surgical and surgical management of peri-implantitis and mucositis.Although not significant, compared to human trials larger treatment effects were reported in animal studies.The authors observed a large degree of heterogeneity between studies which hindered direct comparison and that animal studies were poorly reported [12].
Although not statistically significant, there appeared to be an improvement in reporting of animal studies in relation to the ARRIVE guidelines across the study timeframe.Additionally, the mean score for the essential set (34.2) of items was higher than the recommended set (23.8).This may be an indication of authors prioritizing the reporting on the essential set items.However, the variation in reporting items in both the essential and recommended set may reflect authors reporting items which they perceive relevant/important to their study.
The lack of adequate reporting of animal studies has been attributed to a lack awareness by researchers of the ARRIVE guidelines [18].Furthermore, regional influences have been postulated as a barrier to widespread global adoption of the ARRIVE guidelines, which were devised in the United Kingdom [23].Regardless, it is evident that recommendations to improve the reporting adherence of orthodontic studies using animal models in relation to the ARRIVE 2.0 guidelines are required.These should be directed at all stakeholders including journal editors, authors of studies, peer reviewers, and institutions [24].Measures that can be undertaken by journal editors include insisting upon submission of a reporting checklist with the manuscript and removal of article word restrictions to allow researchers to fully describe all sections [23].Journal editors should note that the active implementation of reporting checklists has been reported to result in improved reporting of studies such as Randomised Clinical Trials within orthodontics [25].To further raise awareness, both journal editors and authors should be signposted to the Equator Network (www.equator-network.org)which aims to enhance the quality and transparency of health research.
The ARRIVE 2.0 guidelines replaced the initial guidelines published in 2010.It may be the case that our results represent some degree of underestimation of the reporting adequacy of the checklist items, as researchers may have been familiar with the initial version of the reporting guideline.In this investigation, animal studies relevant to the speciality of orthodontics published in the wider literature were analysed.This approach was adopted as it is more common for orthodontic studies that utilize animal models to be published in non-orthodontic journals.This is supported by the fact that an initial search of the top five orthodontic impact factor journals only identified 81 studies that met the study eligibility criteria.To reduce any subjectivity, the applicability of the study to the orthodontic speciality was agreed by discussion between two assessors.The possible introduction of bias into the current study maybe further increased by exclusion of non-English studies.However, it should be considered that our aim was to report a baseline of the general standard of reporting which shows there is a lack of adherence to certain items of the ARRIVE 2.0 guidelines.
Measures to reduce bias undertaken in this study were independent and duplicate screening and selection of studies, pre-piloting prior to data extraction, and referring directly to the description of each ARRIVE 2.0 guideline item to ensure

Blinding
Describe who was aware of the group allocation at the different stages of the experiment (during the allocation, the conduct of the experiment, the outcome assessment, and the data analysis).

Outcome measures
a. Clearly define all outcome measures assessed (e.g.cell death, molecular markers, or behavioural changes).0 (0.0) 0 (0.0) 384 (100.0)b.For hypothesis-testing studies, specify the primary outcome measure, i.e. the outcome measure that was used to determine the sample size.consistency in the interpretation of each item during data extraction.Articles published approximately 3 years after the introduction of the latest version of the guidelines were included.We also included articles published 2 years before the introduction of the latest guideline to allow us to observe any trends in reporting.However, it could be argued that an adequate time period has not expired to allow the acceptance and awareness of the ARRIVE 2.0 guidelines by journal editors and researchers.This could be the hypothesis of future investigations.

Conclusions
The reporting of animal studies relevant to the speciality of orthodontics are sub-optimal in relation to the ARRIVE 2.0 guidelines.There was a tendency for the non-reporting of items pertaining to study sample size, eligibility, methods to reduce bias, and interpretation/scientific implications.Greater awareness and reporting adherence to the ARRIVE 2.0 guidelines is required to reduce research waste involving animal models.

Figure 2 .
Figure 2. Radial plot displaying mean adequate responses per ARRIVE 2.0 checklist items.
b. Describe the strategy used to minimize potential confounders such as the order of treatments and measurements, or animal/cage location.If confounders were not controlled, state this Provide the name of the ethical review committee or equivalent that has approved the use of animals in this study, and any relevant licence or protocol numbers (if applicable).If ethical approval was not sought or granted, provide a justification.Describe the humane endpoints established for the study, the signs that were monitored and the frequency of monitoring.If the study did not have humane endpoints, state this.Interpret the results, taking into account the study objectives and hypotheses, current theory and other relevant studies in the literature.Comment on the study limitations including potential sources of bias, limitations of the animal model, and imprecision associated with the results.Comment on whether, and how, the findings of this study are likely to generalize to other species or experimental conditions, including any relevance to human biology (where appropriate).Provide a statement indicating whether a protocol (including the research question, key design features, and analysis plan) was prepared before the study, and if and where this protocol was registered.Declare any potential conflicts of interest, including financial and non-financial.If none exist, this should be stated.

Table 3 .
Estimates, 95% confidence intervals and p-values for the effect of study characteristics and ARRIVE 2.0 summary score.